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Display  2:  Inner  product  representation  of  matrix 
elements  on  the  biplot  of  Display  1 
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In  this  paper,  I  will  discuss  the  biplot  as  a  graphical 
multivariate  technique,  and  I  shall  start  by  showing  what  a 
biplot  is.  I  will  then  explain  and  illustrate  its  use  in  two 
applications:  (1)  in  inspecting  data  matrices  and  (2)  in 

diagnosing  models  to  fit  data.  I  will  end  by  making  some 
comments  on  advantages  of  this  particular  method  as  compared 
to  other  displays  of  multivariate  data. 

THE  BIPLOT 

A  biplot  CGabriel,  1971,  1980)  is  a  graphical  display  of 
a  matrix  Y  of  n  rows  and  m  columns  by  means  of  markers 
-l'-2'*’*'^  for  its  rows  and  markers  b2^»b2,. for  its 
columns.  These  markers  are  chosen  in  such  a  way  that  the 
inner  product  aj^b^  represents  y^  the  i,j-th  element  of  Y. 
Now,  if  we  assemble  all  the  a  markers  as  rows  of  a  matrix  A 
and  all  the  b  markers  as  rows  of  a  matrix  B,  then  this  inner 
product  relationship  means  that  matrix  product  AB*  represents 
the  matrix  Y  itself. 

Let  me  make  a  remark  about  terminology.  The  prefix  ”bi" 
in  "biplot"  does  not  refer  to  its  being  two-dimensional  but 
indicates  that  it  is  a  joint  display  of  rows  and  of  columns 
of  the  matrix  Y.  When  we  have  an  analogous  three-dimensional 
display,  we  refer  to  that  as  a  "bimodel";  the  prefix  ”bi" 
again  Indicates  that  it  is  a  joint  display  of  rows  and  columns 
the  ending  "model"  signifies  that  it  is  not  plotted  in  the 
plane  but  uses  further  dimensions. 

-  Display  1  - 

A  simple  example  of  a  biplot  is  given  in  Display  1.  The 
4  by  3  matrix  Y  can  be  factorized  as  the  product  AB ' ,  A  being 


-2- 


4  by  2,  B*  being  2  by  3.  The  biplot  displays  the  rows  of  A, 
i.e.,  3  £4' AS  well  as  the  rows  of  B,  i.e., 

b^.  The  first  row  of  A,  i.e.,  the  vector  (2,2)  is  displayed  as 
the  point  a^;  the  second  row  (2,1)  is  displayed  as  the  point  £2, 
and  the  other  two  rows  as  points  a^  and  The  coltonns  of  B* 

are  displayed  as  arrows  and  b^*  The  distinction  between 

arrow  display  for  columns  and  point  display  for  rows  is  con¬ 
venient:  The  viewer  immediately  sees  which  are  row  markers 
and  which  are  coltmm  marlcers. 

-  Display  2  - 

The  inner  product  interpretation  of  this  biplot  can  be 
seen  from  Display  2  which  shows  two  of  the  elements  of  Y. 

Element  72^3  represented  on  the  biplot  by  the  inner  product 
of  £2  and  b^.  This  inner  product  can  be  visualized  by  taking 
the  direction  through  vector  b^  and  projecting  the  vector  a2 
onto  it.  The  projection  of  ^2  o>^to  that  direction  is  3 units 
long;  the  length  of  b^  Itself  is  VT*  units  long;  the  product 
is  3/VT'  X  V7~-  3;  hence,  the  inner  product  is  -3,  the  negative 
sign  reflecting  the  projection's  being  in  the  direction  opposite 
to  that  of  the  vector  projected  upon.  Indeed,  element  y2  3  is 
equal  to  -3.  For  another  exaunple,  take  element  y^  3;  The 
inner  product  of  a^  with,  b^  is  visualized  by  projecting  a^ 
onto  the  direction  through  by  (This  is  the  same  direction 
that  was  used  before.).  The  projection  is  of  length.  3/2V37 
the  vector  projected  onto  is  of  length  VT*  ;  they  are  both 
in  the  same  direction;  therefore,  the  inner  product  is 
+3/2V7~  X  V2  *  1  1/2,  which  is  indeed  the  value  of  y. 
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The  matrix  Y  could  be  biplotted  exactly  because  it  was 
of  rank  two.  In  general,  an  exact  biplpt  of  a  matrix  is 
possible  only  if  the  matrix  is  of  rank  one  or  two,  because 
the  biplot  Itself  is  planar.  For  a  matrix  of  higher  rank 
several  steps  have  to  be  taken  in  order  to  display  it  by  an 
approximate  biplot.  The  first  step  is  to  approximate  the 


matrix  Y  by  a  matrix  ^^2]  ^sink  2.  The  second  step  is  to 
factorize  this  rank  2  approximation  ^^2]  ^  product  AB'  of 


a  matrix  A 


(nx2) 


and  a  matrix  B* 


(2xm)  • 


The  third  step  is 


to  take  each  row  of  matrix  A  as  a  row  marker  a  and  each 


column  of  matrix  B'  as  a  column  marker  b.  These  markers  are 


then  plotted  as  an  approximate  biplot  of  the  original  matrix  Y. 

We  next  consider  each  of  these  three  steps  of  approxi- 
mation,  factorization  and  display.  The  best  known  method  for 
lower  rank  approximation  is  due  to  Householder  and  Young  (1938) . 

It  minimizes  the  sum  of  squares  of  the  deviations  of  elements 
of  Y  from  eluents  of  the  reduced  rank  matrix  ^[2J*  However, 
this  method  cannot  be  applied  directly  when  weights  are 
involved.  The  elegant  mathematical  relations  that  were  used 
by  Householder  and  Young  break  down  as  soon  as  one  uses  weighted 

2 

least  squares  and  multiplies  the  squared  deviation  (yi,j~y{2Ji 
by  a  weight  w.  ..  An  algorithm  is  available  (Gabriel  and  Zamir, 

j 

1979) ,  which  allows  this  more  general  approximation.  For  a 
special  kind  of  weights,  Haber  (1975)  found  an  earlier  solution. 
Another  method  of  fitting  lower  rank  matrices  is  by  adaptive 
fits  (McNeil  and  Tukey,  1975) ,  and  yet  further  methods  might 
become  available. 


Factorization  of  the  rank  2  approximation  Y^2]  always 


possible.  Matrices  ®(inx2)  satisfy  Yjjj  ■  AB' 

must  exist.  That  follows  from  the  definition  of  the  rank 
of  a  matrix.  However,  such  a  factorization  is  not  unique.  In 
fact,  if  we  post-multiply  A  by  any  2x2  nonsingular  R  and  pre> 
multiply  B*  by  the  inverse  r“\  the  resulting  (AR) (r“^b' 1  factor¬ 
ization  is  just  as  valid  as  the  original  AB*  factorization.  We 
therefore  have  a  choice  as  to  which  factorization  to  biplot. 

Note  that  transformation  by  a  nonsingular  matrix  consists  of  a 
rotation  of  axes,  a  scaling  along  the  new  axes  and  another 
rotation,  whereas  the  transformation  by  the  inverse  consists 
of  the  same  rotations  with  a  scaling  which  is  reciprocal  to 
the  first  one.  This  may  help  to  give  an  idea  of  how  different 
factorizations  and  different  biplots  are  related.  (An  illustra¬ 
tion  of  alternative  factorizations  and  the  resulting  biplots 
was  given  by  Gabriel,  1971). 

The  non-uniqueness  of  the  factorization  has  some  advantages 
for  the  statistician,  who  may  choose  a  factorization  which  has 
desirable  data  analytic  or  statistical  features.  For  instance, 
one  particularly  attractive  factorization  is  referred  to  as  the 
Gh'  factorization.  This  has  orthonormal  columns  for  G  and 
therefore  satisfies  y'y  -  Hh',  which  is  especially  useful  if 
the  rows  of  Y  represent  Individuals  and  the  columns  represent 
variables.  Then  y'y  is  n  times  the  estimated  variance-covariance 
matrix,  and  so  the  inner  products  of  the  rows  h  of  H  in  a  GH' 
blplot  represent  the  covariances^ and  the  squared  lengths  of 
the  h's  represent  the  variances.  The  cosines  between  h-vectors 
therefore  represent  the  correlations  between  the  variables. 

This  biplot  is  useful  in  many  statistical  applications. 
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INSPECTION  OF  DATA 

Next;  I  consider  uses  of  the  biplot:.  I  will  first  describe  the 
use  of  the  biplot  for  Inspecting  data  matrices.  It  is  partic¬ 
ularly  useful  for  studying  large  data  matrices,  where  eyeballing 
the  large  collection  of  numbers  is  quite  impractical.  Biplot 
display  makes  it  much  easier  to  see  the  main  features  of  the 
matrix.  I  will  illustrate  this  with  a  moderate  size  exeunple 

because  it  is  easier  to  present  that  in  a  paper.  I  should  stress 
that  I  will  not  use  the  biplot  to  analyze  the  data  statistically, 

and  certainly  not  to  test  it  for  significance.  Rather,  I  will 

use  it  for  "looking  at  the  data". 

-  Display  3  - 

Display  3  shows  the  table  of  per  capita  protein  consumption 
in  25  European  countries:  The  rows  are  countries  and  the 
oolunns  are  nine  different  sources  of  protein.  This  matrix 
is  biplotted  in  Display  4  after  the  mean  of  each  column  has  been 
subtracted  out.  The  points, or  row  markers,  represent  countries} 
the  arrows,  or  column  markers,  represent  sources  of  protein. 

This  happens  to  be  a  GH'-blplot  so  that  the  lengths  of  the 
arrows  represent  the  variances  of  the  different  sources  of 
protein  and  the  angles  represent  their  correlations.  The  center 
of  this  blplot  is  at  the  European  mean,  or  centroid,  of  all 
these  sources  of  protein.  The  goodness  of  fit  of  is  of 

the  order  of  0.85;  that  is,  the  biplot  displays  85%  of  the  sum 
of  squares  of  the  mean  centered  data  matrix  Y. 


-  Display  4  - 


Display  3:  European  protein  consvunption 
(grams  per  head  per  day) 
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Display  GH' -biplot  of  European  protein  consumption 
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Looking  at  the  configuration  of  the  nine  sources  of  protein, 
the  most  striking  thing  we  see  is  that  there  is  a  very  large 
variance  for  cereals  and  a  somewhat  large  one  for  milk,  but 
that  the  variances  are  relatively  small  for  all  the  other  sources 
of  protein.  The  correlations  are  also  interesting.  On  the  left- 
hand  side  of  the  plot  are  all  the  animal  sources  of  proteins; 
the  angles  between  them  are  fairly  small,  which  indicates  high 
correlations  between  animal  sources.  Countries  with  high  protein 
consumption  from  meat  appear  also  to  have  high  protein  consumption 
from  eggs,  poultry,  milk,  etc.  The  marker  for  cereals  is  on  the 
right  side  of  the  biplot,  at  em  angle  of  about  180®  to  the  markers 
for  animal  sources.  Evidently,  countries  that  have  a  high  con- 
siimption  of  protein  from  animal  sources  have  relatively  low  con¬ 
sumption  of  cereal  protein  and  vice  versa.  Next,  we  note  the 
markers  for  fruit  and  vegetables  (and  for  fish  (?))  to  be  at 
about  90"  to  both  animal  source  and  cereal  markers.  Apparently 
these  sources  of  protein  are  pretty  much  uncorrelated  with 
animal  and  cereal  proteins. 

It  is  interesting  to  consider  which  countries  are  typical 
of  each  source,  i.e.,  which  countries  have  high  consumption  of 
each  kind  of  protein.  For  that  purpose,  the  row  markers  can  be 
displayed  by  means  of  three  different  symbols  —  1  for  Western 
and  Northern  Europe,  2  for  Eastern  Europe,  and  3  for  Mediter¬ 
ranean  countries.  This  simple  device  makes  it  easy  to  see  that 
Eastern  European  countries  are  on  the  right  of  the  biplot  along 
with  cereals;  these  countries  consume  much  protein  from  cereals. 
Western  and  Northern  European  countries  are  on  the  left  along 
with  markers  for  animal  protein.  Mediterranean  countries  are 
partly  towards  the  bottom  of  the  biplot,  which  indicates  that 


fruit  and  vegetables,  nuts,  and  fish  are  relatively  important 
sources  of  protein  for  them. 

This  example  illustrates  an  important  feature  of  the 
biplot.  It  displays  not  only  the  configuration  of  the  variables, 
i.e.,  of  the  sources  of  protein,  and  the  scatter  of  the  individ¬ 
uals,  i.e.,  of  the  countries,  but  it  also  relates  the  two.  It 
therefore  is  able  to  reveal,  for  example,  not  only  that  con¬ 
sumption  of  cereal  proteins  is  negatively  correlated  with  con¬ 
sumption  of  animal  proteins,  but  --  and  this  is  the  special  feature 
of  the  biplot  —  it  also  identifies  countries  which  are  typical 
users  of  cereal  protein  and  countries  which  mostly  use  animal 
proteins.  This  joint  display  of  countries  and  sources  justifies 
the  use  of  the  prefix  "bi". 

Another  method  of  displaying  particular  groups  of  countries 
on  the  biplot  is  the  use  of  a  concentration  ellipse  for  the 
points  of  each  group  of  interest.  (A  concentration  ellipse  is 
the  two-dimensional  analogue  of  a  mean  ISD  interval:  It  is 
centered  on  the  points'  centroid  and  its  "shadow"  in  any  direc¬ 
tion  is  a  univariate  mean  iSD  interval  for  the  variate  displayed 
in  that  direction;  see  Dempster,  1969,  Ch.  7.)  The  usefulness  of 
this  concentration  ellipse  display  is  in  sviromarizing  a  large 
nximber  of  points  of  each  group  by  a  simple  figure. 

-  Display  5  - 

The  biplot  of  Display  4  is  shown  again  in  Display  5  with 
the  countries'  row  markers  replaced  by  concentration  ellipses 
for  the  three  groups.  This  very  clearly  shows  the  Northern  and 
Western  European  group  to  be  on  the  left,  in  the  animal  protein 
direction;  the  Eastern  group  on  the  right  in  the  cereal  direction 


Display  5:  GH' -biplot  of  European  protein  consumption  with 
concentration  ellipses  for  groups  of  countries 
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and  the  Mediterranean  group  to  have  a  very  elongated  scatter 
in  the  nuts,  fish,  fruit  and  vegetable  directions.  It  is 
obvious  that  the  Eastern  European  group  is  much  more  hetero*> 
geneous  than  the  V7estern  and  Northern  European  group  emd  the 
shape  of  the  Mediterranean  scatter  makes  one  doubt  whether 
that  should  really  be  considered  as  a  single  group. 

-  Display  6  - 

Use  of  concentration  ellipses  is  of  particular  importance 
when  large  sets  of  data  need  to  be  displayed  and  there  are 
more  row  markers  than  can  be  displayed  effectively.  Display  6 
shows  a  biplot  of  breast  tissue  samples  which  were  analyzed  for 
enzyme  activity.  (The  data  are  due  to  Dr.  Russell  Hilf  of  the 
Department  of  Biochemistry  at  the  University  of  Rochester.) 

The  activity  of  several  enzymes  and  other  phenomena  were 
measured  on  each  of  several  hundred  breast  tissue  samples 
which  had  also  been  classified  into  four  diagnostic  groups: 
normal  tissue  CGroup  4) ,  cancerous  tissue  CGroup  1)  and  two 
kinds  of  benign  growths  (Groups  2  and  3) .  When  all  the  700-odd 
points  were  displayed  on  the  biplot,  it  was  very  difficult  to 
distinguish  the  four  groups  of  points.  But  the  biplot  with  the 
concentration  ellipses  of  the  four  groups  —  Display  6  —  is 
much  easier  to  grasp.  One  sees  a  clear  distinction  between 
the  scatters  of  the  cancerous  and  the  normal  tissues;  each 
shows  different  enzyme  activities.  The  two  benign  growth  groups 
are  Intermediate  between  the  preceding  two  in  enzyme  activities. 


It  is  at  times  useful  to  consider  only  the  variance-co- 
yariance  configuration.  Thus,  in  a  GH*  biplot  one  might  omit 


Display  6:  Biplot  of  enzyme  activity  data  for  samples  of 
breast  tissue  with  concentration  ellipses  for 
four  types  of  diagnosis 
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the  markers  for  individuals  (rows)  and  display  only  the 
variables  (columns)  h-markers.  This  will  be  referred  to  as 
a  h-plot.  One  reason  for  wishing  to  ignore  the  individuals 
could  be  that  they  might  be  mere  san^leS/  or  replicates,  from 
a  population  —  and  that  it  is  only  the  population  as  an 
aggregate  that  is  of  interest.  At  times,  one  might  want  to 
use  several  h-plots  and  compare  the  variance-covariance  con¬ 
figurations  of  several  different  populations. 

-  Displays  7  &  8  - 

An  interesting  exaunple  comes  from  the  first  randomized 
rainmaking  experiment  in  Israel.  Days  were  randomly  allocated 
to  have  clouds  seeded  either  in  the  North  or  in  the  Center 
of  Israel.  Displays  7  and  8  are  h-plots  of  the  precipitation 
in  eight  sub-areas  of  Israel  —  Display  7  for  Center-seeded 
days.  Display  8  for  North-seeded  days  (Corsten  and  Gabriel, 
1976) .  The  two  h-conf igurations  are,  at  first  glance,  very 
similar.  At  the  top  of  each  display  is  the  h  marker  for  the 
South,  then  come  the  markers  for  the  three  sub-areas  of  the 
Center  of  Israel,  then, at  the  bottom  of  the  displays,  are  the 
markers  for  the  North  of  Israel,  and  for  the  "buffer  zone" 
between  the  North  and  the  Center.  Both  displays  show  that 
there  was  high  correlation  between  sub- areas  within  the  North, 
average  correlation  among  the  Center  sub-areas  and  rather 
low  correlation  between  the  Center  and  the  North. 


I0RTH(INTERI0R) 
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Oespite  the  overall  similarity  of  the  configurations  of 
North-seeded  and  Center-seeded  days,  some  differences  are 
revealed  by  closer  inspection  of  Displays  7  and  8.  The  most 
striking  difference  is  that  the  correlations  are  considerably 
higher  in  the  North  when  seeding  was  carried  out  in  the  North. 

Also,  when  one  compares  the  lengths  of  vectors  on  the  two 
h-plots,  one  readily  sees  that  the  variances  of  the  Northern 
sub-areas  were  larger  when  the  North  was  seeded  whereas  the 
variances  of  the  Center  sub-areas  were  larger  when  the  Center 
was  seeded.  The  explanation  for  these  findings  may  be  that  the 
effect  of  seeding  was  (.1)  to  make  the  seeded  sub-areas  more 
similar  to  each  other,  and  C2X  to  augment  the  variance  of  rain¬ 
fall  in  the  seeded  area  Cthe  means  were  also  augmented  — 
though  this  is  not  shown  on  the  h-plots 1 . 

-  Display  9  - 

A  somewhat  more  elaborate  exeunple  is  the  data  in  Display  9 
of  three  different  cranial  measurements  of  how  subspecies  of 
anteaters  collected  at  six  geographical  locations  (Reeve,  1940, 
quoted  by  Seal,  1964) .  The  matrix  that  would  be  biplotted  here 
is  the  six  by  three  table  of  the  six  sample  means  of  the  logarithms 
of  the  three  cranial  measurements.  Since  these  are  averages  of 
samples,  it  is  appropriate  when  calculating  their  rank  2  approx¬ 
imation,  to  weight  them  by  the  sample  sizes  and  the  inverse  of 
the  within  sum  of  squares  and  products  matrix.  This  weighting 
is  identical  to  that  used  in  one-way  multivariate  analysis  of 
variance  CGabriel,  1972). 


-  Display  10  - 
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on  the  resulting  biplot,  referred  to  as  a  JK* -biplot  — 
Display  10  —  each  point  represents  a  sample  from  one  location 
and  each  arrow  represents  a  log  characteristic  measured  —  one 
of  the  three  variables.  What  is  immediately  evident  is  that 
the  three  seunples  of  sub-species  Chapadensis  are  very  similar  — 
they  are  very  close  together  on  this  biplot.  The  location  of 
Chiriquensis  and  the  location  of  Instabilis  are  quite  far  from 
these  three  biplot  locations  emd  from  each  other.  Mexicana  is 
located  between  Chiriquensis  and  Chapadensis.  Also,  the  general 
direction  of  the  variables  is  up  and  slightly  to  the  right,  hence 
that  is  the  direction  of  larger  crania.  This  indicates  that 
Instabilis  is  a  smaller  type  of  anteater,  whereas  Chiriquensis, 
Mexicana  and  Chapadensis  are  all  larger.  The  difference 
between  Chapadensis  and  Chiriquensis,  on  the  other  hand,  is 
not  one  of  overall  size  but  one  of  a  contrast  between  the 
different  variables.  Chiriquensis  is  relatively  larger  on 
the  third  variable  —  greatest  nasal  length  —  whereas 
Chapadensis  is  relatively  larger  on  the  first  two  variables. 

The  two  sub-species  are  thus  seen  to  have  different  profiles 
of  the  variables. 

This  JK'  biplot  differs  from  the  GH'  biplots  described 
above:  Amongst  other  things,  weights  were  used  in  fitting  it. 
However,  because  of  the  particular  weights  used,  biplot  dis¬ 
tances  represent  Mahalanobis  distances  between  the  different 
samples.  Thus,  the  Mahalanobis  distances  between  the  Chapadensis 
san^les  are  small;  the  one  between  Mexicana  and  Chapadensis  is 
less  then  that  between  Chiriquensis  and  Chapadensis,  etc. 


Display  10:  JK'-biplot  of  anteaters  by  subspecies  and  cranial  measurements 
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-  Display  11  - 

The  Mahalanobis  distance  is  closely  related  to  Hotelling's 

2 

T  except  that  the  latter  is  scaled  by  the  sum  of  the  reciprocals 

of  the  sample  sizes.  It  is  possible  to  approximate  the  Hotelling 
2 

T  test  between  pairs  of  samples  by  drawing  circles  around  the 
biplot  sample  markers «  where  the  radii  of  the  circles  depend  on 
the  critical  point  used  for  testing  and  on  the  sizes  of  the 
samples  CGabriel*  1972)..  This  is  illustrated  in  Display  11. 

The  interpretation  of  these  "comparison  circles"  is  very  obvious: 
Circles  which  intersect  show  a  non-significant  comparison;  dis¬ 
joint  circles  show  a  significant  comparison.  Thus,  the  three 
circles  for  Chapadensis  overlap  very  much  with  each  other  and 
also  with  the  Mexicans  circle.  This  indicates  that  there  are 
no  significant  differences  between  the  three  Chapadensis  samples, 
nor  between  them  and  the  Mexicans  sample.  Mexicans  is  not  signif¬ 
icantly  different  from  Chiriquensis  either.  However,  Mexicans  is 
significantly  different  from  Instabilis.  In  fact,  Instabilis 
is  found  to  be  significantly  different  from  everything  else, 
and  Chiriquensis  is  also  different  from  Chapadensis.  The 
general  conclusions  would  be  CD  that  Instabilis  indeed  differs 
from  the  rest  of  the  antes ters  and  is  a  smaller  type;  C2)  the 
larger  an tea ters  are  of  at  least  two  groups:  One  containing 
Chiriquensis,  the  other  Chapadensis;  Mexicans  could  belong  to 
either  of  these  groups  —  there  are  no  significant  differences 
which  would  indicate  to  which. 

The  graphical  test  used  here  is  an  approximation  to 
2 

Hotelling's  T  •  In  many  cases  such  a  Gaussian  test  may  not 
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be  valid  and  more  robust  tests  might  be  needed.  It  might, 
for  example,  be  possible  to  carry  out  re-randomization  tests 
directly  on  biplots.  We  are  currently  trying  out  Mielke's  (1976) 
"multiple  ze^onse  permutation  procedures"  for  that  purpose. 

Let  me  stress,  however,  that  I  see  a  very  limited  role  for  ^ 
significance  testing  in  the  exploration  of  such  multivariate 
data.  In  most  multivariate  situations,  we  have  a  fair  number 
of  samples  and  a  fair  number  of  variables;  and  we  are  rarely 
concerned  with  a  test  of  an  overall  null  hypothesis  for  all 
samples  and  all  variables.  Instead  we  usually  want  to  find 
out  what  sort  of  differences  exist  and  between  which  samples 
they  occur.  We  are  tirying  to  explore  rather  than  to  test. 

Multivariate  analysis  is  essentially  an  exploratory 
technique  rather  than  a  confirmatory  method.  Indeed,  by  the 
time  one  gets  to  the  stage  of  confirmation  and  sets  up  a  well- 
defined  null  hypothesis  for  testing,  one  usually  knows  pretty 
well  which  particular  variable,  or  what  linear  combination  of 
variables , one  is  really  interested  in,  so  that  the  testing 
becomes  univariate  and  not  multivariate.  I  submit  that  multi¬ 
variate  analysis  is  principally  exploratory  and  that  techniques 
such  as  the  biplot  are  usually  very  much  more  to  the  point 

than  most  tests  of  significance. 

DIAGNOSIS  OF  MODELS 

Another  use  of  the  biplot  is  that  of  diagnosing  models 
which  will  fit  a  data  matrix.  This  use  is  particularly  Important 
because  statisticians  really  have  very  few  techniques  available 
for  inspecting  a  data  matrix  and  deciding  what  sort  of  model 
will  fit  it.  Statistics  textbooks  have  ample  material  on  how 
to  test  a  model  once  we  have  formulated  it,  but  little  or 
nothing  on  how  to  select  a  model,  except  by  trial  and  error. 
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A  biplot  may  he  used  to  diagnose  a  model  by  looking 
for  a  pattern  on  the  display  and  then  infer  mathematically 
what  model  that  implies  for  the  data  matrix.  For  exeunple,  if 
the  row  markers  are  seen  to  be  collinear,  and  the  column  markers 
are  also  noted  to  be  collinear,  and  the  two  lines  are  at  right 
angles  to  each  other,  one  may  infer  that  an  additive  model  will 
fit  the  data  closely,  i.e.,y.  .  =  a.  +  6^,  for  some  set  of 

j  j-  j 

alphas  and  betas.  If,  for  another  instance,  one  observes 
row  markers  and  column  markers  to  be  on  two  non-perpen¬ 
dicular  lines,  one  can  infer  that  a  concurrent  model  fits 
the  data,  i.e.,  y^j  *  n  +  some  n»  a^'s  and  6j's. 

(This,  by  the  way, is  a  reparametrization  of  Tukey's  degree-of- 
freedom-for-non- additivity  model.)  Also,  if  one  observes  that 
all  markers,  for  both  rows  and  colvimns/are  on  one  and  the  same 
line,  it  is  obvious  the  matrix  is  a  rank  one  and  so  the  model 
is  yjj  -  a^6j_ 


-  Display  12  - 

Display  12  shows  these  and  some  other  rules  of  diag¬ 
nosis  derived  by  Bradu  and  Gabriel  (1978) .  The  first  line 
indicates  that  when  the  row  markers  are  collinear,  the  data 
may  be  fitted  by  a  columns  regression  model.  (This  model  is  due 
to  John  Mandel  (1961) .  It  expresses  each  column  as  a  linear 
regression  on  given  row  effects  a^^).  The  next  line  of  Display  12 
similarly  shows  that,  when  the  column  markers  are  collinear, 
each  row  can  be  modelled  as  a  linear  regression  on  fixed  column 
6's.  When  both  row  markers  and  column  markers  are  collinear. 


Display  12:  Some,  biplot  diagnostic  rules 
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a  concurrent  model  is  diagnosed  (as  noted  above) ,  unless  these 
two  lines  are  at  90^  to  each  other,  in  which  case  an  additive 
model  is  diagnosed  (as  also  noted  above) . 

The  rules  of  Display  12  apply  even  if  some  of  the  biplot 
markers  are  not  on  these  lines.  In  such  cases,  the  diagnoses 
apply  to  the  subtable  of  the  rows  and  columns  whose  markers 
are  collinear.  This  is  quite  a  remarkable  featvire  of  the  biplot. 
It  makes  it  possible  to  diagnose  models  not  only  for  the  entire 
matrix,  but  also  for  any  sub-matrices.  Most  importantly,  all 
these  diagnostic  indicators  are  very  simple.  The  eye  very 
easily  picks  up  a  straight  line,  even  when  it  fits  only  some 
of  the  row  markers  or  some  of  the  column  markers. 

-  Displays  13  &  14  - 

Here  are  some  further  examples.  Bradu  and  Grine  (1979) 
considered  cranial  measurements  for  a  number  of  specimens  of 
fossils  —  Display  13.  This  table  has  a  large  number  of 
missing  values,  so  that  ordinary  techniques  for  fitting  were 
inappropriate.  Bradu  and  Grine  therefore  used  the  algorithm 
developed  by  Gabriel  and  Zamir  (19791  for  weighted  least 
squares  and  introduced  0  weights  for  the  missing  values  and 
unit  weights  for  present  values.  The  resulting  biplot  is 
shown  in  Display  14.  It  is  quite  remarkable  how  closely  the 
row  markers  cluster  around  one  line  and  the  column  markers 
along  another  line.  Using  the  diagnostics  of  Display  12, 

Bradu  and  Grine  inferred  that  a  concurrent  model  would  fit  the 
data  very  closely,  as  indeed  it  did.  We  note,  however,  that 
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Display  13:  Thirty  neasureinents  on  26  specimens  of 
Diademondintine  crania 
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Display  14:  Biplot  of  Diademondintine  crania  data 
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the  angle  between  the  lines  is  very  close  to  90°  and  suspect 
that  an  additive  model  would  also  have  fit  pretty  well. 

-  Display  15  - 

A  more  complex  example  is  data  of  Gamma  radiation 
~  Display  15  —  classified  by  distance  from  the  radiation 
source,  number  of  intervening  plates,  the  metal  of  which 
these  plates  consisted  and  two  replications.  This  is  a  four¬ 
way  layout  so  one  has  to  confound  several  classifications  in 
the  rows  and/or  in  the  columns  before  one  can  display  it  in 
a  biplot,  as  that  is  a  matrix  display.  One  way  of  doing  this 
is  to  consider  the  data  in  the  matrix  form  of  Display  15, 
with  the  metals,  distances  and  replications  confounded  in  the 
rows,  and  only  the  number  of  plates  appearing  in  the  columns. 
This  matrix  —  after  subtracting  the  overall  mean  ~  is 
biplotted  in  Display  16,  in  which  the  column  markers  represent 
numbers  of  plates,  and  each  row  marker  represents  a  combination 
of  metal,  distance  and  replication. 

-  Display  16  - 

The  biplot  —  Display  16  —  of  the  radiation  data  clearly 
shows  a  linear  pattern  for  the  number  of  plates.  The  pattern 
for  the  row  markers  is  not  so  Immediately  obvious.  However,  if 
for  each  of  the  ten  distance  x  metal  combinations,  we  average 
two  replications,  we  find  that  these  ten  average  markers  lie  on 
a  non- rectangular  lattice:  The  metals  form  two  parallel  lines 
and  the  distances  form  another  five  parallel  lines.  What  model 


Display  15:  Absorption  of  gamma  radiation  by  lead  and  aluminvim 
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1.482 

1.476 

1.454 

1.447 

14 

9.0 

1.249 

1.224 

1.204 

1.211 

1.179 

15 

12.5 

0.976 

0.971 

0.966 

0.960 

0.943 

16 

3.8 

1.916 

1.913 

1.884 

1.887 

1.871 

17 

5.2 

1.732 

1.723 

1.698 

1.696 

1.674 

18 

6.0 

II 

1.632 

1.624 

1.592 

1.588 

1.579 

19 

9.0 

1.344 

1.341 

1.312 

1.311 

1.290 

20 

12.5 

i 

1.118 

1.118 

1.106 

1.086 

1.066 

Display  16 t  Biplot  of  gamma  radiation  data:  Aluminum  {A\ , 
(nximber) ,  niirober  of  plates  (PLT) ,  replications 
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can  be  diagnosed  from  such  a  pattern?  It  may  be  useful  to  go 
through  the  algebraic  steps  of  modelling  for  this  case.  Start¬ 
ing  from  any  origin  we  can  model  the  line  for  the  column  markers 
^  »*  £  +  ApS»  where  Xp  is  a  parameter  for  the  number  of  plates  p 
and  8  is  in  the  direction  in  which  the  column  markers  lie. 

We  can  also  model  the  row  markers  for  the  average  of  the  two 
replications  ss  ^  ^  ®  with  parameter  depending 

on  the  distances  d,  and  parameter  on  the  metals  m.  Vector  y 
would  be  in  the  direction  of  the  parallels  for  the  metals 
whereas  £  would  be  in  the  direction  of  the  parallels  for  the 
distances.  To  see  the  form  of  the  model  for  the  data,  we 
take  the  inner-product 

*  ‘♦ax'  *  *  ‘pS* 

■♦dJl'a  +  ♦mi'a  +  ♦d*plS  *  ♦m*pi'£- 

This  models  the  average  y^  „  „  by  an  effect  due  to  distance, 
plus  an  effect  due  to  metal,  plus  two  multiplicative  effects, 
l.e..  Interaction  terms,  one  of  distance  with  plates  and  the 
other  of  metals  with  plates. 

However,  there  is  still  more  to  be  gleaned  from  the 
biplot  of  Display  16.  The  lines  for  lead  and  for  aluminum 
are  virtually  parallel  and  pretty  much  at  right  angles  to  the 
line  for  plates.  In  terms  of  our  parametrizations,  this 
means  the  vector  £  is  orthogonal  to  vector  Therefore, 
the  inner  product  £*£  is  zero  and  that  term  vanishes  from  the 
model.  Defining  “  i.'s.  +  obtains 

the  model  ya^,n,p.  "  ’^d  '^ro'^p*  ’•'to  only  two  values, 

this  results  in  two  additive  submodels,  one  for  each  metal. 
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The  distance  effects  ir^  are  the  same  for  both  metals,  but  the 
number  of  plate  effects  differ  by  a  constant  of  proportionality. 
Indeed  this  kind  of  model  could  be  fitted  to  these  data.  Note 
also  that  this  example  Illustrates  diagnosis  for  subtables: 

Rule  four  of  Display  12  directly  indicates  an  additive  model 
for  the  data  of  each  metal.  (See  Kester,  1979,  for  further  rules.) 

Further  parametrization  could  be  effected  by  noting 
that  the  distances  along  the  lines  through  the  column  markers 
were  pretty  much  proportional  to  the  nvonber  of  plates  and 
therefore  the  parameters  could  be  expressed  as  linear  in 
the  number  of  plates;  similarly,  the  parallel  lines  for  dis¬ 
tances  were  spaced  pretty  much  proportionally  to  the  distances 
from  the  source  of  radiation  and  so  could  be  presented  as  a 
linear,  or  perhaps  more  precisely  as  a  quadratic,  expression 
in  the  distance  d.  The  actual  model  that  was  fitted  was  an 
elaboration  of  the  above  and  included  linear  and  quadratic 
terms  in  the  number  of  plates  and  in  the  distances  from  the 
source  of  radiation. 

This  is  not  only  an  instance  of  successful  modelling  but 
also  shows  the  method  by  which  a  pattern  observed  on  the  biplot 
is  translated  algebraically  into  a  model  for  the  data. 

All  the  models  that  we  have  diagnosed  so  far  have  been  linear 
or  bilinear  in  the  various  effects.  It  may  be  of  interest  to  con¬ 
sider  an  instance  in  which  such  modelling  was  not  sufficient. 

The  example  is  one  of  mean  monthly  temperatures  during  the 
24  months  of  1951  and  1952  at  50  stations  on  the  American 
continents  (Brier  and  Meltesen,  1976)  The  data  were  bi- 
plotted  —  Display  17  —  after  the  average  temperatures  for 
all  50  stations  were  subtracted  out  ~  goodness-of-fit  was  96%. 
(Ihis  analysis  was  carried  out  jointly  with  Mike  Tsianco,  1980.) 
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-  Display  17  - 

There  is  nothing  particularly  revealing  eibout  the  scatter 
of  row  markers  for  the  stations  in  Display  17.  But  the  column 
markers  fan  out  in  a  rather  systematic  manner:  At  first  sight 
they  would  seem  to  be  collinear  and  suggest  a  rows  regression 
model.  However f  the  order  of  the  different  months  is  interest¬ 
ing.  It  reveals  a  very  similar  configuration  for  the  two  years, 
with  January  at  the  top,  then  February  and  December,  then  March 
and  November,  and,  somewhat  farther  down,  April,  October  and 
May,  then  June,  August,  July  and  September.  What  sort  of  model 
does  this  suggest? 

-  Display  18  - 

We  note  that  the  time  sequence  is  systematic,  going  down 
from  January  to  February,  then  to  March  and  further  down  till 
June,  then  going  up  from  July  to  December.  This  suggests  that 
the  time  sequence  may  really  be  three-dimensional,  the  up-down-up 
movement  on  the  biplot  being  complemented  by  a  further  change  in 
a  third  dimension  separating  spring  from  fall.  It  is  therefore 
%forthwhile  to  fit  a  bimodel,  i.e.,  a  three-dimensional  analogue 
of  a  biplot,  and  look  at  the  plane  of  the  second  and  third 
dimension,  that  is,  essentially  inspect  the  entire  configuration 
from  the  right-hand  side.  This  is  shown  in  Display  18  in  which 
the  column  markers  are  displayed  on  the  plane  of  the  second  and 
third  axes  of  the  bimodel.  Now  we  see  a  clear  elliptical  pattern 
from  winter  on  the  right,  through  spring  at  the  bottom,  sunmer 
on  the  left  and  fall  on  top. 


Display  17:  GH* -bimodel  (1st  and  2nd  dimensions)  of  temperature 
data:  Dots  -  stations,  lines  -  months 

39E+03  -.20E4'03  .  00E4^0  .20E-^03  .39E403 
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Display  19:  Projections 
data  (1951, 
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-  Display  19  - 

Another  way  of  visualizing  this  is  by  means  of  ellipses 
fitted  to  the  three-dimensional  h  configuration.  Display  19 
shows  several  projections  of  these  fitted  ellipses  as  viewed 
from  the  front,  that  is  along  the  first  and  second  dimensions; 
as  viewed  from  the  side,  as  we  saw  a  moment  ago;  as  viewed 
from  the  top;  and  as  viewed  orthogonally  to  the  plane  of  the 
ellipse.  (Note  that  in  Display  19  the  individual  months  are 
shown  as  averages  for  the  two  years  and  so  there  are  only  12 
markers  instead  of  the  24  of  Display  18.  Also  note  that  these 
are  not  concentration  ellipses.) 

What  can  be  inferred  from  this  elliptic  pattern  about 
models  suitable  for  the  data?  What  we  are  modelling  by  an 
ellipse  is  the  configviration  of  the  h-vectors  of  the  GH 

I 

bimodel.  Y  is  displayed  by  rank  3  matrix  product  GH  ,  and 
we  are  not  considering  the  G  factor  but  only  the  24  colxunns  of 
H,  each  having  three  elements.  These  h's  have  an  dliptical  config¬ 
uration  which  we  may  model  as  follows,  hj“  +  ocose^  +  ^sinfl^, 
for  three-element  vectors  ]i,  o  and  B,  emd  angles  9^,...,e24< 

This  familiar  parametrization  of  an  ellipse  represents  the 
the  center  by  £,  the  major  axis  by  o,  the  minor  axis  by  B. 
and  the  points  along  it  by  angles  Bj. 

So  far  we  have  a  model  for  the  h's  as  observed  on  the 
bimodel.  But  our  real  concern  is  to  obtain  a  model  for  the 
data  matrix  itself,  i.e.,  for  the  elements  the 

bimodel  representation  Yj^  j  *  ^lllj  ^  Si  ^ 
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one  of  the  vectors  hj .  In  view  of  the  elliptical  modelling 
of  the  latter,  we  obtain  ^  aj^ocosOj  +  2|Bsin0j. 

This  can  be  simplified  by  the  following  reparametri ration: 

'’i  ■  to^cos^i^  s*  and  K»^sin(^^  =  2ii* 

The  model  then  becomes  y,  .  »  -f*  t|>  •  cos<^ .  cos(^ .  +  i|*sini^  •  sin(( . , 

i/j  1  1  1  j  1  3 

and,  by  the  ordinary  laws  of  trigonometry,  that  equals 
y.  .  ■  +  t|».cos(*.  +  0.).  This  simple  harmonic  model  for 

1,  j  1  1  1  j 

the  data  has  thus  been  shown  to  have  been  diagnosed  by  means 
of  the  bimodel. 

This  model  makes  a  lot  of  meteorological  sense,  is  a 
station  average  temperature;  the  amplitude  of  the  annual 
harmonic  variation  in  temperatxire ,  and  when  the  model  was  fitted, 
these  amplitudes  were  found  to  be  larger  farther  from  the  equator 
and  smaller  close  to  the  equator,  as  one  would  expect  (Tsianco, 
1980) .  The  harmonic  cosine  element  has  its  phases  in  terms  of 
two  arguments,  depending  on  the  station  i  and  0j  depending 
on  the  month  j.  Tsianco  found  0j  to  change  from  month  to  month 
by  almost  exactly  2ir/12,  as  one  would  expect  from  the  annual 
cycle  of  temperature.  He  found  the  fitted  values  of  to  be 
much  the  same  for  all  North  American  stations  and  again  much 
the  same  for  all  South  American  stations  —  the  difference 
between  the  Northern  and  the  Southern  was  w  —  which  is 

what  one  would  expect  since  it  is  well-known  that  it  is  warm 
in  the  North  when  it  is  cold  in  the  South  and  vice  versa. 

This  example  has  shown  how  inspection  of  a  biplot/biroodel 
may  lead  to  observation  of  a  pattern  which  can  be  modelled  and 
how  such  a  model  can  lead  to  a  model  for  the  data  themselves. 

It  has  also  sho«m  that  the  resulting  model  is  in  accord  with 
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what  we  know  about  meteorology.  Thus,  biplot/bimodel  inspec¬ 
tion  and  consequent  modelling  for  the  data  may  give  physically 
appropriate  models. 

SOME  GENERAL  COMMENTS 

It  may  be  in  order  to  state  the  sequence  in  which  I  think 
display  and  modelling  should  be  applied.  One  should  begin  by 
fitting  the  biplot  or  bimodel  to  a  data  matrix;  from  inspection 
of  this  display  one  might  be  able  to  infer  a  model  or  formulate 
a  description  of  the  data.  Before  one  could  conclude  that  this 
was  an  appropriate  description,  one  should  look  at  the  residuals 
and  ask  whether  they  might  be  related  to  the  fit  of  the  biplot 
or  the  model,  and/or  whether  they  might  be  heteroscedastic.  If 
so,  one  should  look  for  forms  of  re-expression  in  the  hope  of 
yielding  more  homogeneous,  less  systematic,  residuals  from  the 
next  fit.  This  fitting,  looking  at  residuals,  re-expression 
sequence  should  be  iterated  until  one  is  satisfied  that  the 
residuals  are  mainly  noise. 

Whilst  doing  these  Inspections  of  residuals,  one  should 
not  merely  look  at  general  patterns  of  residuals  but  also  spot 
outliers.  In  fact,  this  would  seem  to  be  an  essential  pre¬ 
liminary  stage  in  all  inspection  of  data.  If  there  are  extreme 
outliers,  one  must  check  the  records  from  which  the  data  came  — 
most  of  the  time  one  would  find  gross  errors  which  need  to  be 
corrected.  In  some  cases,  unexplained  outliers  would  remain. 

It  is  extremely  Important  to  note  unexplained  outlying  residuals 
in  reporting  analyses,  even  if  they  are  omitted  from  the  follow¬ 
ing  fits  and  modelling  because  the  methods  of  fitting  might  be 
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unduly  influenced  by  them.  Scientists  who  are  interested  in 
the  data  often  find  the  outliers  to  be  the  roost  fascinating 
and  instructive  part  of  the  whole  data  set.  We,  as  applied 
mathematicians,  enjoy  finding  patterns  and  fitting  models  and 
get  the  satisfaction  of  mathematical  elegance  of  presentation 
of  these  regularities.  But  this  may  be  of  little  interest  to 
the  scientists  who  are  looking  for  new  and  unexpected  phenomena 
rather  than  for  neat  formulation  of  patterns  with  which  they 
are  already  familiar.  It  may  well  be  that  much  of  the  progress 
of  science  is  in  finding  the  unexpected,  the  outliers,  and 
being  led  to  new  ideas  rather  than  in  systematizing  and  para¬ 
meterizing  the  familiar. 

Let  me  make  some  final  remarks  about  biplot  display  in 
comparison  to  a  number  of  other  techniques  of  data  analysis. 

There  are  a  number  of  steps  in  biplot  display.  (1)  We  start 
with  a  matrix  Y.  C2)  We  compute  a  reduced  rank  approximation 
Yj2]'  factorize  that  as  ^^2]  *  snd  then  (4)  we 

display  the  a's  and  b's  in  a  biplot  (or  bimodel).  Regularities 
that  are  in  the  original  data  can  generally  be  expected  to 
remain  in  the  reduced  rank  approximation  and  therefore  to  be 
expressed  in  the  factorization  and  to  appear  as  patterns  on 
the  biplot.  So  matrix  regularities  will  be  displayed  as  biplot 
patterns.  But  scientific  Inference  must  proceed  in  the  opposite 
direction.  One  observes  the  biplot  patterns  and  tries  to  infer 
about  the  data.  This  is  possible  to  the  extent  that  the  steps 
of  approximation,  factorization  and  display  are  reversible.  Indeed, 
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display  is  reversed  by  visualization  and  factorization  by  inner- 
product  multiplication.  But  the  approximation  step  is  reversible 
only  as  well  as  the  goodness  of  fit  of  the  reduced  rank  approxi¬ 
mation.  Often  these  approximations  are  very  close  and  then  one 

# 

can  say  that  the  steps  back  from  the  biplot  to  the  data  can  be 
retraced  almost  exactly.  A  number  of  examples  have  been  pre¬ 
sented  above  which  show  how  one  may  parameterize  a  relationship, 
or  pattern,  on  the  biplot  and  then  retrace  the  steps  to  see 
what  model  suits  the  data  matrix. 

The  possibility  of  reproducing  the  data,  at  least  approxi¬ 
mately,  from  the  biplot/bimodel  display,  is  a  unique  feature 
of  this  particular  method.  There  are  a  number  of  other  methods, 
such  as  multidimensional  scaling  or  correspondence  analysis,  in 
which  one  starts  from  a  matrix,  calculates  a  function  of  the 
matrix,  e.g.,  interpoint  distances,  correlations,  etc.  and  then 
produces  some  map  of  these  distances  or  correlations  by  metric 
or  non-metric  methods.  If  there  are  regularities  in  the  data 
then  these  maps  of  distances  or  correlations  should  reflect 
them.  But  we  cannot  even  approximately  retrace  the  step  from 
the  map  of  distances,  or  correlations,  to  the  original  data. 

This  is  because  the  distance,  or  correlation,  functions  which 
have  been  used  to  summarize  the  data  are  generally  not  one-to-one 
functions.  Hence  one  cannot  reproduce  the  data.  One  may  model 
the  distances,  or  correlations,  but  one  cannot  model  the  data 
by  any  of  these  other  methods.  The  biplot  seems  to  be  unique 
in  that  it  permits  going  back  the  extra  step  to  the  original 
data. 


J 


In  summary,  two  main  uses  of  the  biplot  have  been  pre'~ 
sen ted.  One  is  to  inspect  data  matrices  and  look  for  patterns 
and  relationships.  In  that  use  the  biplot  is  very  similar  to 
several  other  methods.  The  other  use  of  the  biplot  is  to 
diagnose  models  to  fit  the  data.  For  that  purpose  the  biplot 
seems  to  be  unique. 
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The  blplot  graphical  display  is  introduced  and  its 
mathematical  properties  indicated.  Its  use  for  inspecting 
data  matrices  is  illustrated  for  small  and  large  matrices 
as  well  as  for  variance-covariance  configurations  and  multi¬ 
variate  means  of  several  samples.  Its  usefulness  in  diagnosinc 
models  is  explained  and  illustrated  by  several  examples 
requiring  increasingly  complex  linear  and  harmonic  models,  (ove 
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